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September  27,  2000 
Congressional  Committees 


Subject:  Defense  Transportation:  Final  Evaluation  Plan  Is  Needed  to  Assess 
Alternatives  to  the  Current  Personal  Property  Program 

The  Department  of  Defense  spends  about  $1.2  billion  annually  to  move  the  household 
goods  and  unaccompanied  baggage  of  relocating  servicemembers  and  their  families. 
The  Department  has  long  been  concerned  about  the  quality  of  services  it  provides 
military  personnel  and  their  families  when  they  relocate.  Past  problems  include  poor 
service  from  movers,  excessive  loss  or  damage  to  property,  and  high  claims  costs  to 
the  government.  All  of  these  problems  have  contributed  to  poor  quality  of  service  for 
people  using  the  personal  property  program. 

Currently,  the  Department’s  Transportation  Command  is  developing  an  evaluation 
plan  to  assess  alternatives  to  the  existing  personal  property  program.  The  Command 
originally  expected  to  have  its  plan  in  place  by  spring  1999  and  to  begin  collecting 
data  from  the  current  program  and  two  pilot  projects  no  later  than  January  2000  to 
develop  recommendations  for  a  new  personal  property  program.  These  goals  have 
not  been  achieved  because  the  Department  has  not  yet  concluded  what  methodology 
will  be  included  in  the  evaluation  plan  to  analyze  costs  and  because  one  of  the  pilot 
projects  will  not  be  ready  to  be  assessed  until  late  February  2001.  In  the  meantime, 
the  Command  decided  to  begin  collecting  data  in  June  2000  for  completed  sections  of 
the  evaluation  plan  from  the  current  program  and  the  other  pilot  project,  which  is  in 
its  second  year  of  operation.  The  Transportation  Command  and  the  Military  Traffic 
Management  Command,  the  office  administering  the  ongoing  pilot  project,  have 
awarded  contracts  to  obtain  assistance  in  developing  the  evaluation  plan  and  in 
gathering  and  analyzing  data  for  this  year’s  assessment.  The  Military  Traffic 
Management  Command  has  also  awarded  contracts  to  private  sector  transportation 
companies  that  will  be  moving  the  servicemembers’  household  goods  and 
unaccompanied  baggage. 

This  report  provides  an  update  on  the  Transportation  Command’s  efforts  to  evaluate 
alternatives  to  the  current  personal  property  program.  Our  objective  was  to  assess 
the  Command’s  decision  to  begin  data  collection  efforts  before  the  evaluation  plan 
was  completed.  Our  involvement  in  this  issue  stems  from  the  conference  report1  on 


1  House  Report  104-863  p.  866  (Sept.  28, 1996). 
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the  fiscal  year  1997  Defense  Appropriations  Act  and  a  request  from  the  House  Armed 
Services  Subcommittee  on  Military  Readiness  after  our  testimony2  during  its  March 
18,  1999,  hearing  on  the  Department’s  personal  property  program. 

RESULTS  IN  BRIEF 

The  Transportation  Command’s  decision  to  begin  collecting  data  from  the  ongoing 
pilot  project  and  current  program  in  June  2000  was  reasonable  because  (1)  the 
Command  needs  to  determine  this  year  whether  its  evaluation  plan  can  be 
successfully  executed  so  that  any  necessary  changes  can  be  made  before  any  further 
assessments  of  pilot  projects  are  initiated  in  2001;  (2)  the  current  evaluation  plan, 
although  incomplete,  provides  sufficient  guidance  to  allow  the  Command  to  begin 
collecting  time-sensitive  and  other  data  required  for  analysis;  and  (3)  the  collection  of 
data  during  the  high-demand  summer  moving  season  in  this  year’s  assessment  allows 
the  Department  to  begin  determining  whether  its  pilot  projects  can  provide  the  level 
of  services  desired  for  its  servicemembers  and  their  families  when  they  relocate.  The 
current  evaluation  plan  is  incomplete  because  methodological  issues  related  to  how 
total  costs  will  be  measured  remain  unresolved.  If  the  Command  does  not  finalize  its 
evaluation  plan  and  complete  its  current  assessment,  it  will  be  unable  to  meet  a 
February  15,  2001,  internal  reporting  deadline.  This  will  impact  the  Command’s 
ability  to  make  any  needed  changes  to  the  plan  before  conducting  its  initial 
assessment  of  the  second  pilot  next  year  and  a  reassessment,  if  needed,  of  the 
ongoing  pilot  before  it  expires  at  the  end  of  2001.  If  these  problems  are  not 
addressed  in  a  timely  manner  and  should  assessments  continue  into  2002,  the 
Department  will  need  to  modify  existing  contracts  or  award  new  contracts.  Should 
this  occur,  the  Department’s  efforts  to  improve  its  personal  property  program  will  be 
further  delayed  and  cost  more. 

We  are  recommending  that  the  Department  resolve  outstanding  cost  issues  this  fall 
so  that  the  Transportation  Command  can  have  a  complete  evaluation  plan  in  place  to 
begin  assessing  alternatives  to  the  current  personal  property  program.  In  oral 
comments  provided  on  a  draft  of  this  report,  the  Department  concurred  with  our 
recommendation  and  agreed  to  take  the  necessary  actions  to  complete  its  evaluation 
plan. 

BACKGROUND 


The  Transportation  Command  is  responsible  for  developing  an  evaluation  plan  for 
assessing  alternatives  to  the  Department  of  Defense’s  personal  property  program. 

This  plan  will  be  used  to  assess  two  pilot  projects.  The  Reengineered  Personal 
Property  Program,  managed  by  the  Military  Traffic  Management  Command,  has  been 
under  way  since  January  1999  and  is  scheduled  to  end  on  December  31,  2001.  The 
Full  Service  Moving  Project,  managed  by  a  project  team  reporting  to  the  Assistant 
Deputy  Under  Secretary  of  Defense  (Transportation  Policy) ,  was  scheduled  to  start  in 
January  2000  but  is  currently  not  expected  to  be  fully  operational  until  the  end  of 


2  Defense  Transportation:  Efforts  to  Improve  DOD’s  Personal  Property  Program  (G A 0 /'I  -NSIAD  99-106 
Mar.  18.  19991. 
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February  2001.  A  third  pilot  project,  Navy’s  Servicemember  Arranged  Move,  will  not 
be  assessed  using  the  Transportation  Command’s  evaluation  plan.  Rather,  this  pilot 
project  will  be  evaluated  separately  by  the  Transportation  Command  using  data  from 
the  Navy’s  1999  assessment.3 

The  evaluation  plan  includes  four  assessment  factors:  quality  of  life,  small  business 
participation,  process  improvements,  and  total  costs.  The  specific  criteria  for  the 
first  three  factors  were  established  by  May  2000.  The  fourth  assessment  factor, 
related  to  total  costs,  will  consider  both  direct  and  indirect  costs  for  the  pilot  projects 
and  the  current  program.4  Discussions  among  the  Transportation  Command  and 
other  Department  offices  are  ongoing  to  reach  agreement  on  the  methodologies  for 
the  cost  factor. 

The  Transportation  Command  is  considering  comments  from  three  other  offices  in  its 
efforts  to  complete  the  cost  section  of  the  plan.  The  Assistant  Deputy  Under 
Secretary  of  Defense  (Transportation  Policy)  issued  guidance  to  the  Command 
stating  that  the  evaluation  plan  and  the  first-year  assessment  of  the  Reengineered 
Program  pilot  project  should  use  the  same  direct  and  indirect  cost  methodologies  to 
assess  the  current  program.  The  Command  is  also  reviewing  the  results  of  the 
Department  of  Defense  Inspector  General’s  evaluation  of  the  current  program’s  cost 
methodologies  that  the  Military  Traffic  Management  Command  plans  to  use  to 
evaluate  its  Reengineered  Program  pilot  project’s  first-year  operations.  The  Inspector 
General  sent  its  draft  report,  dated  June  30,  2000,  to  the  Transportation  Command 
and  requested  that  comments  on  the  report’s  findings  and  recommendations  be 
submitted  by  August  29,  2000.  The  Inspector  General  has  agreed  to  the 
Transportation  Command’s  request  for  an  extension  to  September  15,  2000,  to  allow 
the  Transportation  Command  additional  time  to  facilitate  coordination  within  the 
Department  before  submitting  its  formal  comments.  The  third  office,  the  Military 
Traffic  Management  Command,  previously  concurred  with  the  cost  data  elements 
included  in  the  evaluation  plan;  however,  its  Reengineering  Program  pilot  project 
team  recently  raised  concerns  about  the  availability  of  cost  data  as  specified  in  the 
plan. 


3  The  Reengineered  Program  pilot  project  includes  about  18,500  moves  from  installations  in  North 
Carolina,  South  Carolina,  and  Florida.  The  Full  Service  pilot  project  will  involve  about  45,000  moves 
from  installations  in  the  National  Capital  Region,  Georgia,  and  North  Dakota.  Both  pilots  will  include 
participants  from  all  military  services  and  the  Coast  Guard.  The  Navy’s  assessment  of  its  pilot  project 
included  223  moves  from  5  installations  between  January  1998  and  March  1999.  While  government 
personnel  administer  the  current  program  and  two  of  the  pilot  projects,  the  Full  Service  pilot  project 
will  employ  contractors  to  perform  this  function. 

4  Direct  costs  include  charges  for  transportation,  storage,  and  accessorial  services  (e.g.,  number  of 
boxes  used  and  movement  of  large  items  such  as  pianos) .  Indirect  costs  include  program 
management,  counseling,  quality  assurance,  and  financial  transaction  processing  costs.  Direct  costs 
can  be  linked  to  a  specific  shipment;  indirect  costs  cannot. 
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DECISION  TO  PROCEED  WITH  DATA  COLLECTION  WAS 
REASONABLE.  BUT  EVALUATION  PLAN  HAS  NOT  BEEN  FINALIZED 

Although  the  Transportation  Command  has  not  completed  its  evaluation  plan  and 
was  unable  to  assess  both  pilot  projects  and  the  current  program  as  projected  this 
year,  its  decision  to  begin  collecting  data  from  the  ongoing  pilot  project  and  the 
current  program  in  June  2000  was  reasonable.  However,  unless  outstanding  cost 
issues  are  resolved  this  fall,  the  Command  will  be  unable  to  finalize  the  evaluation 
plan  and  complete  this  year’s  assessment  to  determine  whether  any  changes  are 
needed  to  the  evaluation  plan  before  any  further  assessments  are  conducted  in  2001. 
Furthermore,  should  the  delay  cause  assessments  to  continue  beyond  2001,  the 
Department  will  need  to  modify  existing  contracts  or  award  new  contracts. 

Decision  to  Proceed  With  Data  Collection  Provides  Benefits 


The  Transportation  Command’s  decision  to  begin  collecting  data  from  the 
Reengineered  Program  pilot  project  and  the  current  program  in  June  2000  was 
reasonable  for  the  following  reasons: 

•  The  Command  needs  to  collect  and  analyze  data  from  the  ongoing  pilot  project 
and  current  program  this  year  to  determine  whether  its  evaluation  plan  can  be 
successfully  executed  before  the  initial  assessment  of  the  Full  Service  pilot 
project  begins  next  year.  Also,  if  the  evaluation  plan  needs  to  be  revised,  the 
Command  projects  it  will  have  sufficient  time  to  make  the  necessary  changes  and 
reassess  the  Reengineered  Program  pilot  project  before  it  ends  on  December  31, 
2001. 

•  The  evaluation  plan,  though  incomplete,  contains  sufficient  guidance  to  allow  the 
Command  to  begin  collecting  data  required  for  the  completed  sections  of  the  plan, 
including  quality  of  life,  small  business  participation,  and  process  improvements. 
Of  particular  significance  is  the  need  to  conduct  time-sensitive  surveys  of 
participants  in  the  Reengineering  Program  pilot  project  and  of  a  control  group 
representing  users  of  the  current  program  for  the  quality-of-life  factor.5 

•  Data  collected  this  summer  will  allow  the  Department  to  begin  assessing  whether 
its  pilot  projects  can  provide  the  level  of  services  desired  for  its  servicemembers 
and  their  families  when  they  relocate.  It  is  during  the  summer  that  the  demand 
for  moving  services  is  at  its  highest  and  that  the  Department  competes  the  most 
with  other  government  agencies  and  private-sector  users  for  quality  services  from 
the  moving  industry. 


5  To  assess  the  quality-of-life  factor  for  the  current  program,  the  Transportation  Command  plans  to 
survey  1,000  randomly  selected  servicemembers  from  geographic  locations  outside  the  Reengineered 
Program  and  Full  Service  pilot  project  areas. 
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Effects  of  Further  Delays  in  Completing  the  Evaluation  Plan 

If  the  Department  does  not  resolve  the  outstanding  cost  issues  this  fall,  before  data 
collection  efforts  end,  the  Command  will  be  unable  to  finalize  its  evaluation  plan  and 
meet  its  February  15,  2001,  deadline  for  issuing  an  internal  status  report  on  this  year’s 
assessment  of  the  Reengineered  Program  pilot  project  and  current  program.  This  will 
also  impact  the  Command’s  ability  to  determine  whether  changes  are  needed  to  the 
cost  and  other  sections  of  the  plan  in  time  for  its  initial  assessment  of  the  Full  Service 
pilot  project  in  2001  and,  if  required,  a  reassessment  of  the  Reengineering  Program 
pilot  project.  Summer  2001  will  be  the  last  opportunity  to  obtain  meaningful  data 
from  the  Reengineered  Program  pilot  project  before  it  ends  on  December  31,  2001. 

The  Command  has  identified  several  steps  that  it  believes  must  be  completed  to  meet 
the  February  15,  2001,  reporting  deadline.  Currently,  the  Command  is  collecting  data 
on  moves  initiated  and  completed  from  June  1  through  November  30,  2000. 

Extending  the  assessment  period  from  August  31  to  November  30  allows  for  the 
inclusion  of  more  overseas  moves  and  shipments  involving  temporary  storage  (less 
than  60  days).  Throughout  the  assessment  period,  the  Command  plans  to  collect  and 
analyze  data  for  its  report  on  all  four  of  the  assessment  factors  and  will  also  identify 
and  correct  any  problems,  to  the  extent  possible,  to  maintain  the  integrity  of  this 
year’s  execution  of  the  evaluation  plan.  With  the  exception  of  the  final  quality-of-life 
survey  data,  the  Command  plans  to  have  most  of  its  data  collection  and  analysis 
completed  by  December  31,  2000.  Since  the  Reengineered  Program  pilot  project  will 
be  given  60  days  to  compile  quality-of-life  survey  results  after  each  move  is 
completed,  the  Command  anticipates  receiving  the  final  survey  data  by  January  30, 
2001. 

By  February  15,  2001,  the  Command  plans  to  have  all  data  collection  and  analysis 
tasks  completed,  issue  an  internal  status  report  on  its  assessment  of  the  pilot  project 
and  current  program,  and  identify  any  remaining  deficiencies  in  its  execution  of  the 
evaluation  plan.  The  Command  projects  that  it  will  then  have  less  than  4  months  to 
address  these  deficiencies  before  any  further  assessments,  including  those  carried 
out  in  the  summer  of  2001,  could  begin.  If  the  remaining  deficiencies  are  not 
corrected  by  that  time  and  should  assessments  continue  into  2002,  the  Department 
will  need  to  modify  existing  contracts  or  award  new  contracts.  Any  assessments 
beyond  next  year  will  cause  further  delays  and  higher  costs  for  the  Department  to 
continue  its  efforts  to  improve  the  current  personal  property  program. 

CONCLUSIONS 


The  Department’s  decision  to  begin  collecting  data  on  the  Reengineering  Program 
pilot  project  and  the  current  program  was  reasonable,  even  though  the  evaluation 
plan  was  incomplete.  However,  if  the  Department  does  not  resolve  outstanding 
issues  on  how  total  costs  are  to  be  calculated,  the  Transportation  Command  will  be 
precluded  from  finalizing  its  evaluation  plan,  completing  this  year’s  assessment,  and 
correcting  any  remaining  deficiencies  in  the  plan  before  further  assessments  are 
conducted  in  2001.  Until  these  problems  are  addressed,  the  Command  is  also 
precluded  from  developing  recommendations  for  an  improved  personal  property 
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program.  In  the  meantime,  servicemembers  and  their  families  will  continue  to  be 
subjected  to  problems  with  the  current  program  when  they  relocate. 

RECOMMENDATION  FOR  EXECUTIVE  ACTION 


We  recommend  that  the  Secretary  of  Defense  direct  the  Assistant  Deputy  Under 
Secretary  of  Defense  (Transportation  Policy),  the  Transportation  Command,  and  the 
Department  of  Defense  Inspector  General  take  the  necessary  actions  to  resolve 
outstanding  cost  issues  this  fall  so  that  the  Transportation  Command  can  have  a 
complete  evaluation  plan  in  place  to  begin  assessing  alternatives  to  the  current 
personal  property  program. 

AGENCY  COMMENTS  AND  OUR  EVALUATION 


The  Assistant  for  Surface  Transportation  to  the  Assistant  Deputy  Under  Secretary  of 
Defense  (Transportation  Policy)  and  representatives  from  the  Transportation 
Command  and  the  Department  of  Defense  Inspector  General  provided  oral  comments 
on  a  draft  of  this  report.  They  all  concurred  with  our  recommendation  and  stated 
that  they  plan  to  take  necessary  actions  so  that  the  U.S.  Transportation  Command 
can  have  a  complete  evaluation  plan  in  place  this  fall  to  begin  assessing  alternatives 
to  the  current  personal  property  program.  The  officials  also  provided  technical 
comments,  and  we  revised  our  report  to  reflect  them  where  appropriate. 

SCOPE  AND  METHODOLOGY 


To  assess  the  Transportation  Command’s  decision  to  proceed  with  data  collection  for 
the  Reengineered  Program  pilot  project  and  the  current  program,  we  reviewed  the 
latest  version  of  the  Command’s  evaluation  plan  to  determine  to  what  extent  it 
addresses  the  four  assessment  factors:  quality  of  life,  small  business  participation, 
process  improvements,  and  total  costs.  We  met  with  Department  officials  and  their 
contractors  to  discuss  prior  problems  the  Department  encountered  in  studying 
alternatives  to  the  current  personal  property  program  and  the  ongoing  efforts  to 
address  these  problems.  We  discussed  lessons  learned  from  our  review  of  the  Army 
Hunter  Pilot  Project  and  the  criteria  we  would  use  to  assess  the  evaluation  plan.6  We 
also  provided  the  Transportation  Command  with  excerpts  of  our  General 
Policies/Procedures  and  Communications  Manual  on  sampling  methodology  and 
copies  of  two  reference  guides,7  as  requested. 

We  met  with  officials  from  the  Office  of  the  Assistant  Deputy  Under  Secretary  of 
Defense  (Transportation  Policy),  Washington,  D.C.;  the  U.S.  Transportation 
Command,  Scott  Air  Force  Base,  Illinois;  the  Military  Traffic  Management  Command, 
Alexandria,  Virginia;  the  Department  of  Defense  Inspector  General,  Full  Service 


6  Defense  Transportation:  The  Army’s  Hunter  Pilot  Project  Is  Inconclusive  but  Provides  Lessons 
Learned  (GAO/NSIAD-99-129.  Tune  23.  19991. 

7  Developing  and  Using  Questionnaires  (GAQ/PEMD-10.1.7.  Oct.  1993)  and  Using  Structured 
Interviewing  Techniques  (GAQ/PEMD-10.1.5.  Tune  1991) . 
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Moving  Project,  and  Hay  Group  (Transportation  Command  Contractor),  Arlington, 
Virginia;  and  American  Management  Systems  (Transportation  Command  contractor), 
PricewaterhouseCoopers  (Military  Traffic  Management  Command  contractor),  and 
Systems  Research  and  Applications  (Military  Traffic  Management  Command 
contractor),  Fairfax,  Virginia.  We  conducted  our  review  from  February  through 
August  2000  in  accordance  with  generally  accepted  government  auditing  standards. 


We  are  sending  copies  of  this  report  to  the  Honorable  William  S.  Cohen,  Secretary  of 
Defense;  the  Honorable  Louis  Caldera,  Secretary  of  the  Army;  General  Charles  T. 
Robertson,  Jr.,  Commander  in  Chief,  U.S.  Transportation  Command;  Major  General 
Kenneth  L.  Privratsky,  Commander,  Military  Traffic  Management  Command;  and  the 
Honorable  Jacob  J.  Lew,  Director,  Office  of  Management  and  Budget.  We  are  also 
sending  copies  to  interested  congressional  members.  Additional  copies  will  be 
available  to  others  upon  request. 

Please  see  enclosure  I  for  contacts  and  key  contributors  to  this  assignment. 


David  R.  Warren,  Director 
Defense  Management  Issues 


7 


GAO/NSIAD-00-217R  Defense  Transportation 


B-285940 


LIST  OF  CONGRESSIONAL  COMMITTEES 

The  Honorable  John  W.  Warner 
Chairman 

The  Honorable  Carl  Levin 
Ranking  Minority  Member 
Committee  on  Armed  Services 
United  States  Senate 

The  Honorable  Ted  Stevens 
Chairman 

The  Honorable  Daniel  K.  Inouye 
Ranking  Minority  Member 
Subcommittee  on  Defense 
Committee  on  Appropriations 
United  States  Senate 

The  Honorable  Floyd  D.  Spence 
Chairman 

The  Honorable  Ike  Skelton 
Ranking  Minority  Member 
Committee  on  Armed  Services 
House  of  Representatives 

The  Honorable  Solomon  P.  Ortiz 
Ranking  Minority  Member 
Subcommittee  on  Military  Readiness 
Committee  on  Armed  Services 
House  of  Representatives 

The  Honorable  Jerry  Lewis 
Chairman 

The  Honorable  John  P.  Murtha 
Ranking  Minority  Member 
Subcommittee  on  Defense 
Committee  on  Appropriations 
House  of  Representatives 
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ENCLOSURE  I 


GAO  CONTACTS  AND  STAFF  ACKNOWLEDGMENTS 


GAO  CONTACTS 


David  R.  Warren  (202)  512-8412 
Charles  I.  Patton,  Jr.  (202)  512-8412 

ACKNOWLEDGMENTS 


In  addition  to  those  named  above,  Lawson  Gist,  Jr.,  Robert  L.  Self, 

Jacqueline  S.  McColl,  Marjorie  L.  Pratt,  Arthur  L.  James,  Jr.,  Molly  B.  Boyle,  and 
John  G.  Brosnan  made  key  contributions  to  this  assignment. 
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Defense  Transportation:  The  Army ’s  Hunter  Pilot  Project  to  Outsource  Relocation  Services 
(GAO/NSIAD-98-149.  June  10.  1998). 

Defense  Outsourcing:  Better  Data  Needed  to  Support  Overhead  Rates  for  A-76  Studies 
(GAO/NSIAD-98-62.  Feb.  27. 1998). 

Defense  Transportation:  Reengineering  the  DOD  Personal  Property  Program 
(GAO/NSIAD-97-49.  Nov.  27.  1996s). 
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